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INTRODUCTION 

What impact does distance education have on student learning? Is it more effective 
than traditional education? Less effective? Or about the same? Many studies that 
compare the two find no significant differences in learning and other outcome 
measures. The perception is that most studies on distance education or the use of 
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technology are poorly designed and prone to incomplete analyses. That certainly is true 
of the simple comparison study, where student outcomes (such as course grades) for 
an online course are compared to those of a traditional course. This is the source of the 
"no significant differences" phenomenon, where possible intervening forces are ignored. 
Often, the researcher and instructor are the same person, further muddying the results. 
This design is flawed and the results are questionable. 

However, there are some very good studies, some quantitative, others qualitative, and 
still others thoughtful or theoretical analyses of what is occurring in online courses. 

Some of these studies are quite creative and use interesting approaches to analyze the 
online course or the student learning resulting from using the Web in a course. Many of 
these studies would pass the harshest peer review criteria. Others are less complicated 
but no less worth reading. 

It is unlikely we will ever unravel all of the factors that impact online learning. It is 
complex and its elements (the technology and the students) keep changing. Since we 
have not achieved a definitive answer on quality for more traditional classroom 
situations, perhaps it is unwise to expect such clarity for online learning. However, more 
understanding is always better, so the search for clarity will continue. 

THE NO SIGNIFICANT DIFFERENCE PHENOMENON 

Perhaps the most quoted and misunderstood body of research on distanceeducation 
has been the work of Russell (1999), who reviewed 355 studies ondistance education 
produced from 1928 to 1998. Some of the early studiesexamined correspondence 
courses, but most compared instruction overvideotape, interactive video, or satellite with 
on-campus, in-personcourses. Students were compared on test scores, grades, or 
performancemeasures unique to the study, and also on student 
satisfaction. Consistently, based on statistical tests, "no significant difference"between 
the comparison groups was found. However, only 40 of the 355studies specifically 
included computer-based instruction, and thecompilation was completed prior to the 
blossoming of courses using the Web. 

It is important to understand the ramifications of Russell's work. Despite the technology 
used, the results are the same: no difference in student achievement. Russell 
concludes, "There is nothing inherent in the technology that elicits improvements in 
learning," although "the process of redesigning a course to adapt the content to the 
technology" can improve the course and improve the outcomes" (p. xiii). In other words, 
learning is not caused by the technology, but by the instructional method "embedded in 
the media" (Clark, 1994, p. 22). Technology, then, is "merely a means of delivering 
instruction," a delivery truck, so to speak, that does not influence achievement. Russell 
concludes, "No matter how it is produced, how it is delivered, whether or not it is 
interactive, low-tech or high-tech, students learn equally well" (p. xiv). Russell 
expressed his frustration that, after so many studies, people continue to believe that 
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technology impacts learning. 

OTHER COMPARISON STUDIES 

Surprisingly, a large number of studies reviewed for the ASHE-ERIC Report upon which 
this Digest is based still compare student achievement between web-based versus 
in-person delivery models. Not surprisingly, the results of studies by Bourne, McMaster, 
Rieger, and Campbell (1997), Davies and Mendenhall (1998), Dominguez and Ridley 
(1 999), Gagne and Shepherd (2001 ), Hahn et al. (1 990), Johnson (2001 ), McNeill et al. 
(1991), Miller (2000), Mulligan and Geary (1999), Ryan (2000), Schulman and Sims 
(1999), Sener and Stover (2000), Serban (2000), Wegner, Holloway, and Garton 
(1999), Wideman and Owston (1999) remain largely the same as in Russell's 
compilation: comparing the two types of delivery methods leads to a conclusion of no 
significant difference in student achievement. However, several of these studies found 
differences in completion or student satisfaction, although final grades or exam scores 
were often the same, or nearly the same, between the two types of courses compared. 

If the comparison studies (Russell, 1999) accomplished anything, they established that 
the technology studied did not make as much of a difference in the selected learning 
outcomes as some expected. This is due to the fact that interactive video (two-way 
audio and video conferencing) may sufficiently duplicate the traditional classroom 
teacher-centered model as to be indistinguishable from that model. Its instructional 
model is "one-to-many" whether delivered in person via lecture, television, or via 
interactive video. Or as Morrison (2001) remarked, "If you try to compare media, you 
have to keep the instruction constant. If you keep it constant, and the medium does not 
change the message/instruction, you will find no significant difference." 

OTHER STUDY RESULTS 

Two studies are unique for their use of control variables. Kuh and Vesper (1999) 
analyzed data on 125,224 undergraduates and found that to the extent that the students 
became familiar with computers, there was a significant and positive association with 
self-reported gains in self-directed learning, writing, and problem solving (this study is 
unique for also having controlled for such factors as grades, age, gender, parental 
education, and educational aspirations). Another study by Flowers, Pascarella, and 
Pierson (2000) modeled on the Kuh and Vesper research focused on cognitive impacts 
of computer use during the first year of college. These results did not duplicate the 
positive results of Kuh and Vesper (1999), and while the impact on students at four-year 
colleges was nonsignificant, the results for community college students were positive, 
indicating a difference in the type of student enrolled in the two settings or their 
experiences while enrolled. Positive results were found for use of word processing in 
reading comprehension. 

AREAS FOR FURTHER RESEARCH 
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While many aspects of using the web have been investigated, other issues have not. 
Research is needed into the usefulness or appropriateness of the web for different 
disciplines or learning objectives. Fahy (2000) calls this "technology's fitness for use" as 
a teaching tool, and asks - as others have asked -- whether the technology is directly 
related to the learning outcome. Are some technologies more appropriate for 
visual-based disciplines and others better for discourse as Tuckey (1993) contends? Is 
the web good for lower-division courses, but inadequate for graduate seminars? And 
finally, what is the "best media mix" to achieve different learning goals (Harasim, 1996)? 
Or, as Barbules and Callister (2000) put the challenge, "Which technologies have 
educational potential for which students, for which subject matters, and for which 
purposes." 

There may be an emerging answer to the series of "which" questions posed by 
Barbules and Callister. In early studies of K-12 students studying science reviewed by 
Helgeson (1988), the most effective combination of instructional opportunities included 
hands-on laboratory experiences and computer simulations, improving students' 
scientific thinking. This study is one of the first that drew attention to the possibility that a 
mix of media may be the most powerful means of education. Campos and Harasim 
(1999) found 55% of students prefer mixed-mode classes: those that combine 
face-to-face and online activities. Young (2002) describes "hybrid" teaching (or the "the 
convergence of online and resident instruction") at several universities, which one 
university president calls "the single-greatest unrecognized trend in higher education 
today." Dziuban and Moskal (2001) found that courses with both a web and face-to-face 
component produced the same or better success rates than courses that were fully 
online or face-to-face. This result teases us into asking whether there is some optimal 
combination of technologies - not limited to face-to-face, interactive video, and Web - 
that maximize learning based on the needs of the curriculum, the type of learning 
desired, and learner characteristics. Over time, the correct question to ask may not be 
which is better, but what combination is best. 

SUMMARY 



Much of the research on Web-based courses (whether these are comparison studies or 
case studies) indicates that students do as well or better and are satisfied with their 
learning experiences. Ample interaction (with material, students, and faculty) and 
constructivist learning situations (e.g., project- and problem-based learning) enabled by 
the Web may be the key to this improved performance. But student learning may also 
depend on a number of individual qualities, including a positive attitude and motivation, 
independence and sufficient computer skills, as well as a predominantly visual learning 
style and an understanding that learning is not a passive process of absorbing 
information. These individual differences will make it difficult to promote any one 
approach as good for everyone. 
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The mission of the ERIC system is to improve American education by increasing and 
facilitating the use of educational research and information to effect good practice in the 
activities of learning, teaching, educational decision making, and research, wherever 
and whenever these activities take place. 
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